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EVALUATION OF THE RIGHT TO READ SPECIAL EMPHASIS PROJECT 

The Special Emphasis Project represented a unique experimental 
intervention in the teaching of reading to elementary students experiencing 
difficulties learning to read. The summary of the evaluation of this 
project is presented in the following four sections: 

• The Special Emphasis Projecf— A brief discussion of the 
background and provisions of the Special Emphasis Project. 

• The Evaluation of Speci al Emphasis— A description of the 
evaluation study. 

• The Evaluation Findings— A summary of process and impact 
findings resulting from the evaluation study. 

• Conclusions . 

THE SPECIAL EMPHASIS PROJECT 

The Right to Read Special Emphasis Project was initiated under 
the provisions of Public Law 93-380, Section 721, as amended by Public 
Law 94-194, Section 10. This legislation authorized the US Office 
of Education (USOE) to undertake a study to test the hypothesis that 
intensive programs of reading instruction introduced at an early afi 
can change patterns of reading achievement of students in schools 
having large numbers of students reading one or more grades below 
level. The USOE originally funded eight projects in the fall of 1976 
for the purpose of implementing the Special Emphasis concept. Each 
project was to provide fors 

• The teaching of reading by reading specialists for all 
children in grades 1 and 2. 

• The teaching of reading by reading specialists for children, 
in grades 3 through 6, who had reading problems. 

• The provision of an intensive vacation reading program 

fcr elementary school children who were reading below 

the appropriate grade level or who were experiencing problems 
in learning to read. 



In addition to these major requirements, each Special Emphasis 
project was to incorporate programmatic features as specified in the 
authorizing legislation. Among these features were: 

• A diagnostic-prescriptive approach to reading instruction. 

• Inservice training opportunities for project school stsff. 

• Parent involvement in the program. 

• Cooperation with an external evaluation. 

To facilitate the evaluation of the Special Emphasis Program, USOE 
also required each funded project to identify a "comparison" school - 
which was comparable to the Special Emphasis project school in size, 
student characteristics, instructional approaches, and curriculum 
materials. This school was to be used as the control school in a 
quasi-experimental evaluation design. 

Eight Special Emphasis projects were funded in 1976. Of these, 
six were refunded, and one new project was funded for school years 
1977-78 and 1978-79. The seven Special Emphasis projects which operated 
in 1977-78 and 1978-79 are included in this evaluation. These projects 
were located in Louisiana, Michigan, Ohio, Tennessee, Texas, West 
Virginia, and California. 

THE EVALUATION OF SPECIAL EMPHASIS 

The following paragraphs describe the objectives and the limita- 
tions of the Special Emphasis evaluation. 

Objectives 

Concurrent with the funding of the Special Emphasis projects, 
the USOE awarded an evaluation contract. The contractor for this 
evaluation was required to conduct an evaluation of: 

• The process by which the Special Emphasis programs were 
implemented and operated. 

• The impact of the Special Emphasis projects on reading 
achievement, on reading attitudes and behaviors, and on 
local education agency (LEA) practices or policies. 
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The specific objectives of the process evaluation component 
of this study were: 

• To examine the degree to which each Special Emphasis project 
adhered to project guidelines. 

• To determine the comparability of schools and students 
in the Special Emphasis and comparison schools. 

• To describe the Special Emphasis instructional program 
during both the school year and the summer session. 

The process evaluation was conducted using data on instructional 
programs and data oh student and staff characteristics. Process evalua- 
tion data used in this study were collected from all participating 
staff, students, and parents using questionnaires administered in 
the spring of each project year. In addition, evaluation contractor 
staff interviewed project personnel and observed reading instruction 
at each project school. D.?.ta were collected over a 3-year period. 
Data were analy2.ed on a site-by-site basis and across sites. 



were: 



The objectives of the impact evaluation component of this study 

• To determine the impact on reading performance of students 
in the Special Emphasis and comparison schools. 

• To determine the impact on reading-related attitudes and 
behaviors of school staff, students, and parents. 

• To document any residual effects of the Special Emphasis 
project within the participating school district and on 
project participants. 

• To determine the differences in the retention of reading 
achievement between schools with, and without, 

reading programs. 

The impact evaluation component of this study, based on change 
in student reading performance over time, was to be measured and contrasted 
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at the Special Emphasis and comparison schools. The measure used 

to assess performance was the comprehension total ^ score on the Stanford 

Diagnostic Reading Test (SDRT). Students were tested each fall and 

spring. Questionnaires and interview responses provided additional 

data to meet the impact evaluation objectives. Data were examined 

for each Special Emphasis site to determine impact for each project 

year and over the duration of the project. 

Limitations 

In conducting the evaluation of the Special Emphasis Project, 
several design and analysis problems were encountered. The most signifi- 
cant of the problems limiting the results of the study were: 

• The lack of comparability between some Special Emphasis 
and comparison schools with respect to one or more of 

the following: size, student characteristics, instructional 
approaches, and materials. 

• The lack of comparability among project sites. 

• The existence of other intervention programs, such as 
Title I, in Special Emphasis and/or comparison schools. 

• The limitations of the reading test selected for use in 
the study; the SDRT is a diagnostic rather than an achieve- 
ment test, it uses different norming referent groups for 
fall aad spring test administration. 

• The loss of almost half of the potential observations 
owing to factors such as absenteeism and student turnover. 

• The wide-spread prhsence of ceiling effects on student 
test scores. 

• The lack of suitable comparison groups for use in the 
analysis of the impact of the sunmier programs, due to 
the voluntary nature of these programs. 



^Comprehe nsion total is a score derived from two subtests on the SDRT. 
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The original evaluation plan envisioned by USOE called for a 
study based on croSsS site aggregated data. However, this plan was 
not undertaken wh'ta .n.de disparities were found across project sites. 
Site to site dif cer .aces in criteria for Special Emphasis Program 
participation, lit ;;ual treatment, and in student characteristics 
(e.g., race, socioeconomic status), existed. In view of this, it 
was decided to treat each project site as an independent evaluation, 
that is, impact was analyzed on a site-by-site basis rather than across 
sites. It was also determined that an analysis of summer program 
impact could not be undertaken due to the lack of comparison groups. 

THE EVALUATION FINDINGS 

Findings of the evaluation of the Special Emphasis Project are 
grouped under two general headings: 

• Process findings 

• Impact findings 

Process Findings 

The major findings to emerge from the analysis of descriptive 
data collected on Special Emphasis are summarized below. 

c All project sites secured the services of reading special- 
ists or reading teachers, whose qualifications met the 
USOE requirements for Special Emphasis. 

• All but two of the sites served all students in grades 

1 and 2; all but one of the sites provided Special Emphasis 
instruction to those students, in grades 3 through 6, 
who experienced problems in reading. 

• The minimum requirement of 40 minutes per day of reading 
instruction was met by all but one of the sites. 

• • All sites conducted summer vacation school reading instruc- 

tion programs. 

• All sites conducted inservice training for Special Emphasis 
staff and classroom teachers. 
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• All sites established record keeping systems to track 
student mastery of reading skills. 

• Only one site had project and comparison schools which 
were comparable with respect to instructional approaches, 
curriculum materials, size of enrollment, and student 
charac teris t ics . 

• None of the Special Emphasis projects was in compliance 
with all legislated and USOE established requirements. 

The process evaluation identified three Special Emphasis projects 
which implemented programs that closely resembled the Special Emphasis 
concept and came closest to meeting the established requirements. 
Data from these project sites, when compared with data from the other 
four sites, established two program features that were conmon to the 
three. 

• • Reading specialists worked in a classroom teaming with 

the regular classroom teacher. Genuine teaming required 

(1) a period of adjustment to establish roles and ''turf," 

(2) an opportunity to plan the instruction and to discuss 
individual student performance (i.e., a planning period), 

(3) guidance and coordination by the project leader, and 

(4) a common instructional resource. Where reading special- 
ists were not teamed with the classroom teacher within 

the classroom, conferences were less frequent, annoyances 
more common, and little or no instructional coordination 
occurred. 

• Inservice training was an integral element in the program. 
It was a key enabling mechanism when it (1) established 

a conmon instructional approach for teachers and specialists 
(i.e., diagnostic-prescriptive), (2) provided teachers 
and specialists the opportunity to discuss and resolve 
project related issues, and (3) provided teachers and 
specialists the opportunity to create curriculum components 
and instructional materials which met the specific needs 
of the project school. 



A review of the data collected from each of the seven Special 
Emphasis sites revealed one other point of general interest. Local 
articulation of site-specific project objectives focused on process 
rather than outcome aspects of the project. Only one site expressed 
objectives in terms of measurable outcomes; two other sites expressed 
one objective as a measurable outcome. 

Impact Findings 

From the outset of the project, the effect of Special Emphasis 
on student reading performance, as measured by scores on a standardized 
test, was •comnonly regarded as the major criterion on which program 
impact was to be measured. However, while the evaluation of reading 
achievement gains was the majo^ focus of this study, other outcomes 
from the Special Emphasis project were also considered. The follow- 
ing paragraphs discuss the results of each of the analyses pc-rformed 
including: 

• Analysis of Change in Reading Performance for Whole Grade 
Cohort Groups Using Analysis of Covariance (ANCOVA) . 

• Analysis of Change in Reading Performance for Below Mean 
Cohort Groups Using ANCOVA. 

• Trend Analysis of Students Reading I or More Years Below 
Grade Level. 

• Attitudinal and Behavioral Change. 

• Residual Effects. 

Based upon the results of the process findings and onsite data 
collection activities and impressions, the study team anticipated 
that three project sites provided the greatest potential for Special 
Emphasis impact findings to emerge. As previously noted> these sites 
implemented programs that closely conformed to project regulations 
and guidelines. The results of the impact analysis tended to support 
expectations that project impact was linked with project implementation. 
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Project sites represent one of tvo implementation groups; Group I 
consists of the three sites with the highest qualitative point totals 
relative to the implementation of the Special Emphasis requirements 
(Louisiana, Tennessee, and Texas) and Group II consists of the remaining 
four project sites (Michigan, Ohio, West Virginia, and California). 
Table 4,46 in Volume I presents data regarding the implementation ratings 
for each project site on which these groupings are based. 

Analysis of Change in Reading Perf ormr nee for Whole Grade Cohort 
Groups Using ANCOVA 

ANCOVA is a statistical technique used Lo test differences among 
two or more groups while controlling foi initial group differences on 
mitigating variables. This technique was used to compare student 
reading performance at each grade at each Special Emphasis school and 
its respective compi^rison school. The post test comprehension total 
scaled scores of each grade cohort group (those students in an individual 
grade who took both pre- and post tests) were adjusted by the difference 
in pretest comprehension total scaled scores between the Special Emphasis 
and comparison groups. Following this adjustment, variations in posttest 
scores between groups were investigated for statistical significance. 

Table 1 provides a sijmmary of instances of statistically significant 
differences -^-i reading performance between Special Emphasis and comparison 
whole grade cohort groups. 

Within Group I sites, Louisiana and Texas had significant differ- 
ences for whole grade cohort groups in the analysis. At the Louisiana 
site, the ANCOVA perforated for whole grade cohort groups revealed 
statistically significant differences between the Special Emphasis 
and the comparison groups in 1^/7-78 and 1978-79 at grades 2 and 5. 
Results favored the Special Emphasis groups in both cases. At the 
Texas site, the ANCOVA favored the Special Emphasis groups in 1977-78 
and 1978-79 at grade 2. Tennessee, despite its adherence to programmatic 
regxilations and guidelines, showed no significant differences between 
Special Emphasis and comparison groups. These two groups, however, were 
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poorly matched, a fact which may have influenced this finding. Isolated 
instances of significaAt differences between whole grade cohort groups 
were also observed in Group II sites, Michigan, and West Virginia; some 
favor Special Eaqjhasis, others the comparison group. 



TABLE 1 

ANCOVA WHOLE GRADE COHORT GROUPS SUMMARY 



CROP? I 



Grads 

2 


1977-78 
SE 


LOUXSIAIM 
1978-79 

SB 


i 

1977-79 
M/A 


1977-78 


TENNESSEl 
1978-79 


; 

1977-79 


1977-78 
SE 


TEXAS 
1978-79 

SE 


1977-79 
K/A 


3 






SE 














4 




















) 


SB 


SE 


N/A 










SE 


SE 


6 






C 















G? OUP II 



Cradt 

2 


1977-78 


HICHIGAN 
1978-79 


1977-79 
N/A 


OHIO 
1977-78 


1977-78 


fEST VIEGI 
1978-79 

C 


1977-79 
N/A 


CALIFDRJfiA 
1978-79 


3 




C 












»/A 


4 










C 


SE 


SE 




5 




C 






C 




C 




6 


i 


H/A 


!I/A 














— 



Key: SE ■ Significant dlffenncc. Special Emphasis group 

C » Significant difference, conparlaon group 

*■ No significant difference 

N/A ■ Data tiot avn liable 



Analysis of Change in Reading Performance for Belov Maan C ohort 
Groups Using ANCOVA ^ 

The covariance analysis for below mean cohort groups was similar 
to that described above except that the analytic samples were subsets 
of the whole grade groups. Below mean cohort groups consisted of 
those students who had pretest scores below the mean pretest score 
for their respective whole grade cohort group. 

The below mean analysis attempted to investigate program Impact 
on students for whom Special Emphasis was targeted. There are several 



hazards associated with the V3e of ANCOVA for below mean cohort groups 
(the proportion of students in Special Emphasis and comparison groups 
scoring belcw the mean are not equal in several cases, a preponderance 
of negative measurement error at the low end of the distribution exists). 

Table 2 suimnarizes the ANCOVA rr':sults for below jman cohort groups. 



TABLE 2 

ANCOVA BELOW MEAN COHORT GROUPS SUMMARY 



Grade 


LOUIS 


lANA 


TENNESSEE 




1977-78 


197»-79 


1977-78 


1978-79 


JL977-73 


1978-79 


2 


SE 


SE 






SB 


SE 


3 


















SE 










5 




SE 










6 




SE 











CROUP II 





MICE 


:iGAN 


OHIO 


WEST VIRGINIA 


CALIFORNIA 




1977-78 


1978-79 


1977-78 


1977-78 


1978-/9 


1977-78 


1958-79 


2 
















3 


C 








C 




C 


4 








C 


SE 


SE 




5 




C 






SE 






6 






SE 


c J 









SE • Significant difftrenca, Sp««:lal EMphaals 

C ■ Significant dlff«r«nca, conparlaon group 

• ■ No significant dltference 

M/A - Data not aval labia 



Statistically significant differences between below mean cohort 
groups were more frequent than for whole grade cohort groups. Group I 
sites revealed statistically significant differences in Louisiana in 
1978-79 at grades 2, 4, 5, and 6, all favoring Special Emphasis. At 
the Texas site, statistically significant differences in 1977-78 and 
1978-79 at grade 2 were found favoring the Special Emphasis groups. 
Again, no significant differences were found at the Tennessee site. 
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Instances of significant differences between Special Emphasis and 
comparison groups within sites classified as Group II were scattered 
among schools, grade levels, and project years. 

Trend Analys is of Sta J?. nts Reading 1 or ^^Gre Years Below Grade 
Level ' 

The ANCOVA findings were based on changes in reading performance 
for whole grade and below mean cohort groupings. Another measure of 
impact on reading performance was to investigate changes in the percentage 
of students reading 1 or more years below grade level at Special Emphasis, 
and comparison schools. 



At each Special Emphasis and comparison school except California, 
the data show what appeared to be a smaller percentage of the entire 
student body reading 1 or more years below grade level in 1979 than 
in 1977. Close investigation of this phenomenon revealed that the 
apparent overall improvement may have been a function of sam^^e attri- 
tion rather than program participation. Although not typical, a few 
cases of students moving from the more than 1 year below grade cohort 
group to the at grade level (or above) cohort group were observed. 
Decreasing percentages instead were due to students leaving the analytical 
sample and not being tested at a subsequent test point. In addition, 
incoming second grade students in all but the California Special 
Emphasis school performed better on the standardized test used in this 
study than their predecessor group. Tennessee and West Virginia 
represented an exception, a net decrease in the percentage of students 
reading 1 or more years below grade level did occur between 1977 and 
1978 at the comparison schools. 

The trend analysis also revealed that as cohort groups move 
to succeeding grade levels, increasing percentages of students perform 
1 or more years below grade level. This was reflected in 74 of the 88 
cohort groups observed at all Special Emphasis and comparison schools 
over the course of this study. This finding supports the generally 
held belief that student performance declines as grade levels increase, 
e.g., greater numbers of students fall behind as they move to higher 
grade levels . 
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Attltudinal and Behavioral Change 

The impact of Special Emphasis was also measured by investigating 
the effect of the program upon the attitudes and behaviors of participants. 
Questionnaire and survey responses from teachers, students, and parents 
were examined for changes in reading-related attitudes and behaviors 
at each site and school. 

As measured by teacher perceptions, student reading-related 
attitudes and behaviors showed continual improvement in both Special 
Emphasis and comparison schools. The results of surveys of students 
and parents regarding reading attitudes and behaviors were inconclusive.. 

Teacher attitudes toward Special Emphasis were also explored. 
Teachers at Special Emphasis schools who reported problems with the 
Special Emphasis program were generally concerned with an apparent 
overemphasis on reading to the detriment of other program areas. 
This concern increased over the course of the Special Emphasis project. 

Residual Effc tcts 

The continuation of Special Emphasis program features and concepts 
after the termination of Federal funding was regarded as another measure 
of project impact. Interviews conducted at each project isite at the 
conclusion of the Special Emphasis project probed the carry over of 
the concepts, practices, and procedures initiated under Special Emphasis 
to other reading programs. The greatest degree of carry over appears 
to have taken place in Louisiana, Ohio, Tennessee, and Texas where 
programmatic features of the Special Emphasis projects have been 
institutionalized or adapted for LEA use. It should be recalled that 
three of these sites ranked highest in implementing Special Eii^uasis. 
These were also the sites in which school district officials showed 
the greatest interest in the project. Moreover, at these sites. 
Special Emphasis was recognized as a unique educational intervention 
effort. Teachers claim that their practices will continue to reflect an 
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Instructional approach based on skill assessment and skill building in 
small 'tlassroom groupings (1 to 9 students) as introduced or reinforced 
by the Special Emphasis program. 

CONaUSIONS 

The working hypothesis of the Special Emphasis Project was that 
intensive programs of reading instruction introduced at an early age 
would effect significant improvement in patterns of reading achievement 
in schools having large numbers of low achieving students. The data 
collected and analyzed from the seven project sites for this evaluation 
study reveal that Special Emphasis sites themselves can be classified 
according to the degree to which they inipleiTiented the Special Emphasis 
concept .-md program. Of the three sites w"*th the highest implementation 
ratings (Louisiana, Tennessee, and Texas), two showed evidence of impact 
favoring Special Emphasis student groups. The data suggest that program 
implementation had a determining influence on program success and con- 
sequently on student performance. Of the sites achieving lower imple- 
mentation ratings (Michigan, Ohio, West Virginia, and California), 
significant differences between Special Emphasis and comparison student 
groups were scattered between the two student groups. Because it is 
questionable that Special Emphasis was in fact operationaiized at 
these four sites, the few instance.^ of impact favoring Special Emphasis 
groups cannot be construed as evidence of program success. 

Highly operationaiized Special Emphasis programs had two program 
features in common. Reading specialists worked in the classroom teaming 
with the regular classroom teacher and inservice training programs 
attempted to integrate and reinforce a diagnostic-prescriptive approach 
to reading instruction. 

Sites which esdiibited the greatest degree of carry over of Special 
Emphasis concepts and practices when Federal project funds terminated 
were those with the highest implementation ratings. The greatest decree 
of carry over was experienced at the classroom level. Key factors 
associated with the integration or adaptation of Special Emphas**- program 
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features at individual project sites were: an interest and participation 
by the local education agency in the project, recognition of Special 
Emphasis as a unique intervention effort, and a sense of accomplishment 
and involvement on the part of teache-s and reading specialists in 
the project. 

In suiranary, those sites which (1) most closely followed the 
program guidelines, (2) teamed the reading specialist in the classroom 
with the regular classroom teacher and, (3) developed a high degree of 
staff participation in the program through inservice training, experienced 
the greatest impact on student reading scores. Practical experimental 
problems including the confounding influence of other programs, lack 
of comparability between treatment and comparison schools, and the 
lack of precision of the measurement instrument may mask the true 
efficacy of the Special Emphasis concept. 
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